# NPU user guide

***Copyright © Quectel Wireless Solutions Co., Ltd. 2026. All rights reserved.***

---

# What is NPU

NPU (Neural Processing Unit) is the dedicated AI inference hardware integrated in the RK3576, with a computing power of 6 TOPS (INT8). It executes the core computations of neural networks (convolutions, matrix multiplications) in hardware in parallel, 10~25 times faster than the CPU.

# What Can the NPU Do

| **Application** | **Description** |
| --- | --- |
| Object detection | Identify objects in the image (people, cars, cats/dogs...) |
| Image classification | Determine which category an image belongs to |
| Face recognition | Detection / comparison / feature extraction |
| OCR | Text recognition |
| Pose estimation | Identify human keypoints |

On the M2 board, running AI models (such as YOLO object detection) uses the NPU for accelerated inference. This guide walks you through it step by step.

# Onboard Resources (Out of the Box)

The M2 Debian system comes with a complete NPU toolchain:

| **Resource** | **Path** |
| --- | --- |
| NPU driver | rknpu v0.9.8 |
| Inference runtime | /usr/lib/librknnrt.so (v2.3.2) |
| Inference test tools | /usr/bin/rknn_benchmark, rknn_common_test |
| Prebuilt model | /usr/share/model/RK3576/mobilenet_v1.rknn |

# Scenario 1: Run an Image Classification Model (Quickest Start)

**Step 1: Verify NPU is ready**

```bash
# NPU driver version
cat /sys/kernel/debug/rknpu/version
# Expected: RKNPU driver: v0.9.8
```

**Step 2: Run mobilenet classification inference**

```bash
# Run one inference with the prebuilt mobilenet_v1 model + test image
rknn_common_test /usr/share/model/RK3576/mobilenet_v1.rknn /usr/share/model/dog_224x224.jpg 3
```

**Step 3: View results**

The output shows model info, inference time, and classification results. M2 measured mobilenet_v1 single-frame inference is about **2.1ms (470 FPS)** — this is the effect of NPU acceleration; the CPU takes more than 10x longer for the same model.

# Scenario 2: Run YOLO Object Detection (Most Common)

Full YOLO detection steps are in the [YOLO On-Device Deployment for Object Detection] document. Brief here:

```bash
# Run on board (in the model directory)
cd /usr/share/model/yolov5
rknn_yolov5_demo /usr/share/model/yolov5/RK3576/yolov5s-640-640.rknn \
  model/bus.jpg resize /tmp/bus_detected.jpg
```

Measured: 19ms per frame (about 50 FPS), detecting 4 persons + 1 bus.

# Scenario 3: NPU Performance Test (Multiple Models)

```bash
# Onboard performance test tool
rknn_benchmark /usr/share/model/RK3576/mobilenet_v1.rknn
```

RK3576 NPU measured performance (rknn_benchmark):

| **Model** | **Input Size** | **Avg Time** | **FPS** | **Type** |
| --- | --- | --- | --- | --- |
| resnet18 | 224x224 | 3.50ms | 285 | Image classification |
| resnet152 | 224x224 | 19.35ms | 51.7 | Image classification |
| yolov5s | 640x640 | 17.37ms | 57.6 | Object detection |
| yolo11s | 640x640 | 33.24ms | 30.1 | Object detection |

yolov5s at 57 FPS > 30 FPS (video frame rate) means the RK3576 can do real-time video object detection.

# Scenario 4: Run Your Own Model (Advanced)

Convert your model (PyTorch/TensorFlow/ONNX) to RKNN format to run on the NPU.

**Step 1: Install conversion tools (PC side)**

```bash
cd <SDK>/external/rknn-toolkit2/rknn-toolkit2
pip3 install packages/x86_64/rknn_toolkit2-2.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
pip3 install "onnx==1.14.1"
```

**Step 2: Export ONNX model**

```python
import torch
model = torch.hub.load('pytorch/vision:v0.14.1', 'resnet18', pretrained=True)
model.eval()
torch.onnx.export(model, torch.Tensor(1,3,224,224), 'resnet18.onnx',
                  input_names=['input'], output_names=['output'], opset_version=12)
```

**Step 3: Convert to RKNN**

```python
from rknn.api import RKNN
rknn = RKNN()
rknn.config(mean_values=[123.675,116.28,103.53], std_values=[58.395,58.395,58.395],
            target_platform='rk3576')   # ★ Must specify rk3576
rknn.load_onnx(model='resnet18.onnx')
rknn.build(do_quantization=True, dataset='dataset.txt')
rknn.export_rknn('resnet18.rknn')
```

**Step 4: Push to board and run**

```bash
adb push resnet18.rknn /data/local/tmp/
adb shell "rknn_benchmark /data/local/tmp/resnet18.rknn 2>&1 | grep -i avg"
# Expected: Avg Time 3.50ms, Avg FPS = 285.7
```

Reproducible verification package: rknn-pytorch-demo.zip in the SDK root directory (includes resnet18/resnet152/yolo11s conversion scripts + one-click test).

# FAQ

| **Problem** | **Solution** |
| --- | --- |
| failed to open model | The model must be converted for the rk3576 platform; check the path |
| rknn_benchmark not found | On SMARC (Buildroot), use rknn_common_test instead |
| onnx.mapping error during conversion | onnx version must be ≤ 1.14.1 |
| Slow inference | Fix NPU frequency to max 950MHz (see below) |

# NPU Frequency Locking (Recommended Before Running Models)

```bash
# Root permission, lock NPU to max frequency
echo userspace > /sys/class/devfreq/27700000.npu/governor
echo 950000000 > /sys/class/devfreq/27700000.npu/userspace/set_freq
```